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ABSTRACT 

This research brief describes the findings of a study to 
determine which of the High Schools That Work (HSTW) practices appear to be 
most effective in raising student achievement. The study explored the effects 
of six clusters of practices: (1) curriculum standards; (2) academic and 

career/technical integration; (3) teacher practices; (4) instructional goals; 
(5) guidance counseling of students; and (6) work-based learning. The 
analysis incorporated data from test scores and surveys from the 1996 and 
1998 HSTW assessments of 424 schools involved in the program. Following are 
some of the key findings. The career-oriented high school graduates gained in 
achievement as more of them committed to a solid academic core, even after 
controlling for socioeconomic and racial variables. On average, schools 
raised the proportion of students completing the recommended mathematics 
curriculum by 16 points and the science curriculum by 17 points. Aggregating 
student scores and other information at the school level, on average, there 
was a gain in achievement of 13 points in math, 9 points in science, and 4 
points in reading. Work-based learning, measured by internship participation 
and the use of outside experts to review student work, appeared to have a 
negative effect on achievement, especially in mathematics. (WFA) 
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Key Findings 

T he results of the study demonstrate that from 1996 to 1998 the career-oriented high school 
graduates from 424 high schools became higher achieving as more students completed a 
solid academic core even after controlling for socioeconomic and racial variables. 

■ On average, schools raised the proportion of students completing the recommended mathe- 
matics curriculum by 16 points (from 59 percent to 75 percent) and the science curriculum 
by 17 points (from 35 percent to 52 percent). These increases had a large impact on achieve- 
ment gains in science, mathematics and reading — especially in mathematics. 

■ Aggregating the student scores and other information at the school level, on average there 
was a gain in achievement of 13 points in mathematics, 9 points in science and 4 points in 
reading. These were small, but not inconsequential, gains. 

■ Work-based learning, measured by internship participation and the use of outside experts 
to review student work (a fairly rare occurrence), appeared to have a negative effect on 
student achievement — especially in mathematics. 
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C hallenging course work, good working rela- 
tionships between academic and career/ 
technical teachers, and better communication 
among students and guidance counselors are 
associated with higher test scores in schools that 
belong to High Schools That Work, a network 
focused on raising the academic achievement of 
career/ technical students. 

Some of the major findings that emerged from 
the exploratory study. High Schools That Work and 
Whole School Reform: Raising Academic 
Achievement ofVocational Completers through the 
Reform of School Practice, are summarized in this 
brief This study^ was commissioned by the 
Southern Regional Education Board to determine 
which of its High Schools That Work {HSTW) 
practices appear to be most effective in raising stu- 
dent achievement. //57A17stands out as one of the 
few school-improvement consortia that have 



emphasized data collection and analysis in their 
programs. Since //57T17 began in 1987, the net- 
work has tested students in mathematics, science 
and reading every other year, and it has surveyed 
students and teachers on course-taking patterns, 
attitudes and characteristics in an effort to contin- 
ually measure the success of the effort. 



SREB and the more than 1,100 schools in 
the HSTW network know how well their stu- 
dents perform on the tests and which courses 
they take. But determining which components 
of the program seem to be more effective than 
others required a different kind of study — one 
that would help SREB analyze its school reform 
practices to find quantitative evidence to support 
specific approaches. The research presented here 
— while limited — does provide a basis for fur- 
ther study about how to isolate effective educa- 
tion strategies. 
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The Study Method 

This study explored the effects of the following six clusters of HSTW practices on student achievement: 

■ Curriculum standards ■ Instructional goals 

■ Academic and career/technical integration ■ Guidance counseling of students 

■ Teacher practices ■ Work-based learning 



T hese clusters represent the core of HSTWs philosophy: that 
students concentrating in a career/ technical area will 
benefit from higher expectations in their academic studies 
and that all students should be given sound career guidance and 
opportunities to relate their learning to the world of work. 

Schools in the HSTW network implement practices that are 
closely related to one another, such as setting high expectations 
for students and replacing a general curriculum with a solid aca- 
demic core curriculum. This analysis looks at the association of 
the six independent clusters noted above with school improve- 
ment, after controlling for such variables as the percentage 
of the schools minority students. 



This study is limited since it was not a scientific study that 
evaluated control and experimental groups. The data used to 
assess HSTW practices were already collected, and all schools in 
the //571^sample are theoretically receiving the same treat- 
ment. Furthermore, the data represent the performance of dif- 
ferent groups of students, not one group of students over time. 
For those reasons, readers are cautioned against over-interpreting 
these findings. It cannot be stated with certainty that one prac- 
tice clearly supports gains in student achievement and should be 
favored above others. 



Examining the Data 



T he analysis incorporated test score and survey data from the 1996 and 1998 //57W Assessments of 424 schools involved in the 
network. Aggregating the student scores and other information to the school level, the study found an average achievement 
gain of 13 points in mathematics, 9 points in science and 4 points in reading. These are small, but not inconsequential, gains. 
In fact, the 13-point gain in mathematics is one full standard deviation, or a 16 percent increase. 



Looking at the clusters individually, 

the study found that taking the 
recommended HSTW curriculum — 
essentially college-prep academics and 
high-quality career/ technical studies 
in grades 9-12 — was associated with 
gains in test scores. So were increased 
communication between students and 
their counselors and students’ percep- 
tions that their academic and career/ 
technical teachers were working together. 
Work-based learning, measured by intern- 
ship participation and the use of outside 
experts to review student work, appeared 
to have a negative association. The 
remaining clusters (teacher practices and 
instructional goals) seemed to have little 
substantial measurable effect. 

■ The Curriculum Students Take 

Matters 

The //57*U^recommended curriculum 
calls for at least three credits each in 
mathematics and science. Two of the cred- 
its in each subject must be at the college- 
preparatory level. 



The percentage of students who com- 
pleted the recommended coursework in 
science and mathematics increased sub- 
stantially between 1996 and 1998. On 
average, schools raised the proportion of 
students completing the recommended 
mathematics curriculum by 16 points 
(from 59 percent to about 75 percent) 
and the science curriculum by 17 points 
(from 35 percent to 52 percent). 

These increases had a large impact on 
achievement gains in science, mathematics 
and reading — especially in mathematics. 
A 1 -point change in the percentage of 
students completing the recommended 
mathematics curriculum was associated 
with a 1.1- to 1.6-point increase in sci- 
ence, mathematics and reading test 
scores. For example, if a school increased 
by 10 percent the proportion of students 
completing the recommended curriculum, 
the data suggest that school also would 
increase its average test score by about 1 1 
to 16 points (depending on the subject 
being tested). 



■ Academic and Career/Technical 
Integration Appears to Be an 
Effective Strategy 

Integrating more rigorous academic 
subject matter into career/ technical educa- 
tion is one strategy for reaching students 
who do not thrive in traditional academic 
settings. HSTW\\ 2 lS long made this a 
main component in its reform effort. To 
determine how much a change in the level 
of blended instruction affected test scores, 
the researchers looked at items on the 
1996 and 1998 student surveys that asked 
for students’ perceptions of how well their 
academic and career/technical teachers 
worked together to help them learn math, 
reading and science. 

According to the students, most HSTW 
schools were engaged in a high degree of 
academic and career/ technical integration 
in 1996. About 71 to 75 percent of the 
students said their academic and career/ 
technical teachers worked together to 
improve their academic skills. Between 
1996 and 1998, this cooperation appeared 



erIcv 



4 



to increase even more — between 2 and 6 
percentage points. 

This perceived increase in the level of 
academic and career/ technical teacher col- 
laboration had a great effect on student 
achievement. A 1-point increase in the 
proportion of a school’s students who 
said their academic and career/ technical 
teachers worked together to help 
improve their math skills resulted in 
a predicted 1.2-point increase in mathe- 
matics achievement. In reading, that 
same 1 -point increase translated into 
an increase of also about 1.2 points. 

m Guidance Counseling Also Helps 

Data from the student surveys also sup- 
plied the researchers with a way to meas- 
ure the impact of guidance counseling on 
achievement. If students said they spent 
more time talking to their guidance coun- 
selors between 1996 and 1998, would 
that result in academic gains? This is an 
important question because SREB strong- 
ly believes both guidance counselors and 
teachers play an essential role in advising 
students on career planning and in 
encouraging them to take more challeng- 
ing courses that will help them achieve 
their ultimate goals. 

The study found a strong association 
between achievement gains and career 
guidance: schools in which students 
talked more often with their teachers 
and counselors about their high school 
program increased their average scores. 
The reverse was also true — schools with 
students spending less time talking with 
their teachers and counselors showed 
decreases in test scores. 

A I -point change on the 3-point scale 
for the average amount of time talking with 
counselors (from “not at all” to “somewhat” 
or from “somewhat” to “a great deal”) was 
associated with a significant 8- to 1 1 -point 
increase in academic achievement. 

■ Changes in Instructional Goals 
Were Too Small to Have 
Substantial Effects 

Changing the content and instructional 
delivery practices of teachers also is a 
major goal of HSTW. The network places 
great emphasis on meeting the needs of all 
students through challenging academic 

o 




and career/technical courses that actively 
engage each student in the learning 
process. To study the effect of this cluster 
on student achievement, the researchers 
reviewed answers on 1996 and 1998 
teacher questionnaires that asked about 
the level of importance of seven goals — 
such as helping students in their social 
development, helping them think critical- 
ly, and helping them make plans for what 
they will do after graduation. 

On average, teachers’ answers varied lit- 
tle between 1996 and 1998, essentially 
because nearly all teachers felt all the goals 
were either “very important” or “impor- 
tant.” For this reason, it was difficult to 
note any relationship between instruction- 
al goals and student achievement. 
However, a higher ranking of the 
importance of one goal — helping stu- 
dents pursue a program of study geared 
to their post-high-school plans — 
resulted in an increase of 4 to 5 per- 
centage points in math scores. 

■ Changes in Teaching Practices 
Were Inconclusive 

Changes in the way teachers 
approached instruction between 1996 and 
1998 did not have an appreciable impact 
on student achievement, largely because 
very little changed. The teacher question- 
naires from those years included an item 
that asked teachers to rate whether their 
emphasis in certain areas, ranging from 
; amount of homework assigned to using 
I hands-on experiments in class, had shifted 
I since their school became a member of 
HSTW. Response categories ranged from 
1 {much less) to 5 {much more)^ with 3 
representing no change. 

( On the 5-point scale used for these 
I variables, average change was less than 1/4 
! of one point. Even the maximum change 
I was only about 1-1/2 points. Given this 
! stability in teacher perceptions of their 
; practices, it was not surprising to find lit- 
tle association between this cluster and 
student achievement. 

I m Work-based Leamisig May 
I Negatively Impact Academic 

! Achievement 

//57Wsupports the philosophy that 
many students learn more effectively with- 



in a “real-world” context — that is, within 
a “structured system of work-based and 
school-based learning” that involves 
schools and local community employers. 
Again using the student surveys as a basis 
for the analysis, the researchers reviewed 
the responses of students to questions 
about their participation in internships 
and how often their teachers had an out- 
side expert review their work. 

About a third of the career/ technical 
completers participated in a work based- 
internship (about 31 percent in 1996; 
about 35 percent in 1998.). On the other 
hand, having career/ technical work or 
projects reviewed by outside experts was 
rarer, relatively speaking. Students in most 
schools said this never occurred or that it 
happened only once or twice a year. 

Holding the demographic variables 
constant, increases in the proportion of 
work-based internships and increases in 
the time career/ technical teachers spent 
using outside experts to evaluate student 
work were associated with decreases in 
achievement in mathematics. These find- 
ings suggest that schools that are imple- 
menting work-based training or intern- 
ships should proceed carefully because 
these efforts may not pay off in terms of 
gains in achievement. 

SREB has found in other analyses that 
the following work-based experiences are 
associated with higher student achieve- 
ment: observing and assisting veteran 
workers; being taught how to do the 
work; having the employer instruct stu- 
dents on developing good work habits; 
and having the employer instruct students 
on developing good customer relations. 
Students who had all four of these work- 
based learning experiences had higher 
average achievement scores than did those 
who did not have this combination of 
experiences. Merely having a job is not 
enough to improve a student’s academic 
achievement; however, the quality of the 
worksite experience determines the mag- 
nitude of the impact. 
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Conclusion 



T his Study found that increases in the proportion of students meeting HSTW curriculum standards had a positive impact on 
achievement gains in reading, science and mathematics. Changes in the proportion of students perceiving that their academic 
and career/technical teachers were working together to improve students’ mathematics, reading and writing skills had a positive effect 
on achievement gains. Likewise, increases in the amount of time that students spent talking to their guidance counselors and teachers 
about their school program were directly associated with increases in the schools’ mean assessment scores. 

Although the other broad clusters seemed to have little or no power for predicting school changes in student achievement, educa- 
tors should not ignore certain practices just because they were not specifically linked to higher student achievement. Neither should 
the results of this limited analysis be over-interpreted. The study did not look into how well the practices were actually carried out, 
and the data were not collected for the purpose of an overall evaluation of the theoretical underpinnings of the HSTW reform effort. 

In any analysis of schools, the direction of cause and effect can be difficult to determine. In general, the findings of the study are 
fairly consistent with other research that has found links between certain practices and higher achievement. In the hands of an effec- 
tive educator and in the right school environment, these practices can affect student achievement positively. Perhaps this study and 
related research will spark others to collect similar data and further explore how well schools carry out the practices. These efforts will 
help not only the High Schools That Work initiative but all of those concerned with education to determine what works — and what 
does not — to increase the academic achievement of all students. 



This publication is supported by funds from the Wallace-Reader’s Digest Funds, the Edna McConnell Clark Foundation, the U.S. 
Department of Education and The Goldman Sachs Foundation. The opinions expressed here do not necessarily reflect the positions 
or policies of any of the funding entities, and no official endorsement should be inferred. 
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